Apache Spark

• https://spark.apache.org

Docs

• http://spark.apache.org/docs/latest/index.html
  • http://spark.apache.org/examples.html

Books

• Advanced Analytics with Spark: Patterns for Learning from Data at Scale
• Learning Spark: Lightning-Fast Big Data Analysis

Videos

• Youtube Apache Spark channel
• What’s coming for Spark in 2015 - Bay Area Spark User MeetUp
• A Deeper Understanding of Spark Internals - Aaron Davidson (Databricks)

Courses

• BerkeleyX: CS190.1x Scalable Machine Learning
  • https://github.com/spark-mooc/mooc-setup
• datacamp: introduction to pyspark

Articles

• Large scale matrix multiplication with pyspark (or — how to match two large datasets of company names)

Tips

Spark session and Spark context

You can think of the SparkContext as your connection to the cluster and the SparkSession as your interface with that connection. – https://campus.datacamp.com/courses/introduction-to-pyspark/getting-to-know-pyspark?ex=4

Computing resources

IU

• Spark on Karst at IU